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REMARKS 

Reconsideration and allowance of th§ subject patent application are respectfully 
requested 

An Abstract on a separate sheet is provided. Headings have been added to place the 
specification in a more traditional U.S. format. 

Claims 37 and 39 have been amended to refer to claim 36 as kindly pointed out in the 
office action. 

Claims 29-32 and 35 were rejected under 35 U.S.C. Section 112, second paragraph, as 
allegedly being indefinite. 

Claim 29 has been amended to indicate that each stored probability is multiplied by the 
product of selected stored probabilities, one from each of the neighbouring sampling points. 

As regards claim 35 and the dependency of the number of iterations on the distance 
between salient points in the image, Applicant notes, for example, that the non-limiting, 
illustrative algorithm is multi-scaled. In other words, in a first step the algorithm is applied 
initially not to every pixel, but to a sub-sampled grid of pixels (e.g., every 3 pixels - see page 15, 
line 24 to page 16, line 2). Then subsequently it is applied to finer sampling of the image. If the 
number of iterations were kept the same, then the computational load increases for fine scales, 
while the propagation of information from neighbouring pixels is decreased (as iterations 
continue the neighbouring pixels became more and more "influenced" by pixels further away). 
Consequently, it is better to adjust the number of iterations depending on the scale (i.e., 
depending on the distance between sampling points). In one example implementation, for each 
scale, a fixed, user-defined number of iterations is set. This is system-dependent and 
application-dependent and is estimated, for example, from local knowledge and heuristics. For 
example, in one embodiment, as you move to each finer scale the number of iterations is 
doubled. The claim is therefore directed to the idea that the number of iterations is fixed in 
accordance with the distance between points (i.e., the scale), as opposed to, for example, 
continuing until there is no further change, or until a defined amount of change has happened. 
Applicants submit that in the context of the claim, the language of claim 35 is clear. 
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For these reasons, withdrawal of the Section 1 12, second paragraph, rejection is 
respectfully requested. 

Applicants gratefully acknowledge the indication that claims 31 and 32 contain allowable 
subject matter. 

Claims 24-30, 35-39, 42 and 43 were rejected under 35 U.S.C. Section 103(a) as 
allegedly being unpatentable over the Cham et al. document in view of Martens (U.S. Patent No. 
6,157,677). While not acquiescing in this rejection, claims 24, 42 and 43 have been amended to 
even further emphasize the differences between these claims and the Cham et al. document and 
Martens. The amendments effectively conform these claims to those in the now-granted 
European counterpart EP 1 163 644 Bl. 

First, as set forth in these claims, for each of a plurality of sampling points in each image, 
a plurality of candidate movements are stored, together with the estimated probability of each of 
those candidate movements. Thus, a plurality of candidate movements is maintained and stored 
for each of the sampling points throughout the iterative process. The method does not (as in the 
prior art) involve discarding the less probable movements as the calculation proceeds* 

Second, in performing the recalculation of the probability of each of the candidate 
movements, the claimed method and apparatus looks at the neighbouring sampling points, selects 
one of the plurality of candidate motion probabilities for each of those neighbouring points, and 
uses those to update the candidate movement probabilities at the sampling point in question. 
Therefore, at each of the neighbouring sampling points, there are a plurality of candidate motion 
probabilities from which a selection is made. 

Thus, in the claimed context, the probability of each candidate motion at each sampling 
point is updated by a probability derived from neighbouring sampling points. However, it is not 
the probability of the same movement at neighbouring points which is taken, but instead there is 
a selection of which movement to choose from each neighbouring point. In an illustrated 
example embodiment, it is the maximum probability within the candidate movements at the 
neighbouring sampling point (or, more particularly, the maximum of those which are judged to 
be "similar" to the candidate movement at the sampling point under consideration). This opens 
up the possibility, therefore, that the "weighting" at one sampling point is being carried out by a 
probability of a different motion at a neighbouring sampling point. This can be very important 

9 

PAGE 12/17 * RCVD AT 7/22/2005 7:54:40 PM [Eastern Daylight Time] * SVR:USPTO-EFXRF-6/25 * DNIS:2738300 * CSID:703 81 6 4100 * DURATION (mm-ss):04-10 



NIXON & VANDERHYE PC Fax:703-816-4100 



Jul 22 2005 20:00 



P. 13 



BRADY et al 

Application No. 09/936,998 

Response to Office Action dated February 22, 2005 

for detection of non-rigid body motion because in such a body neighbouring regions may be 
moving in different ways. It therefore represents a significant advantage and allows for 
compensation for non-rigid motion with a greater degree of accuracy. 

The features of the amended independent claims are not found in the cited documents of 
Cham et al. or Martens, or any of the other documents cited. Furthermore, the features are 
significant in detecting movements in non-rigid bodies. As discussed in the introductory part of 
the present application, movement detection is difficult in images which lack recognisable 
features (such as clear edges or corners) and particularly in images where there is non- 
conservative flow (i.e. the total amount of brightness in the image changes). Thus techniques 
which are useful for rigid bodies, or conservative images, do not work well for non-rigid bodies. 

As recognized in the office action, Cham et al. is concerned with matching images of 
rigid bodies. In particular, Cham et al. is concerned with "mosaicing" together fragments of 
images of a complete scene. This is clear, for example, from the illustrations of Figures 3 and 4 
of Cham et al. Thus, the technique of Cham et al. starts from the basis that it is possible to match 
features in the images which are to be mosaiced. This is clear, for example, from page 442, 
right-hand column, second paragraph of Cham which states "[tjhe large differences in the 
viewing directions of the images indicate that a feature-based approach is preferred," It is 
emphasized at page 444, right-hand column, section 2.3 which starts "[sjuppose a number of 
features are extracted from two images and paired as (fli, fri), i = 1, », N " and this section goes 
on to indicate that "[t]he features may also be of various classes (e.g. corners, lines, regions)." 

A comparison of Figures 3 to 6 of Cham et al„ which indicate the sort of images and the 
mosaicing which Cham et al. is attempting, with Figures 2 and 3 of the present application, 
which illustrate the motion detection problem with which the claimed subject matter relates, 
clearly demonstrates the completely different problem which the claimed method and apparatus 
attempt to solve. Cham et al. addresses the problem of wide-baseline matching of images for the 
purposes of image mosaicing, i.e., creating one large panoramic image from a series of smaller 
ones. Wide-baseline means that a large transformation exists between one image and the other. 
Cham et al. combines and extends a number of standard techniques, principally Random 
Sampling and Consensus (RANSAC) and Kalman filtering. 



10 

PAGE 13/17 * RCVD AT 7/22/2005 7:54:40 PM [Eastern Daylight Time}* SVR:USPTO-EFXRF-6/25 * DNIS:2738300 * CSID703 816 4100 " DURATION (mnvss):04-10 



NIXON & VANDERHYE PC Fax:703-816-4100 



Jul 22 2005 20:00 



P. 14 



BRADY et al 

Application No. 09/936,998 

Response to Office Action dated February 22, 2005 

The RANSAC method works as follows: (1) apply a feature detector to images, e,g., a 
corner detector, (2) randomly select a minimal sub-set of such feature pairs, and calculate a 
transformation for these (e.g., four pairs are needed for a 2-D projective transformation), and (3) 
score this transformation by measuring how many of the remaining features fit the proposed 
transformation. These steps (1) to (3) are repeated many thousands of times and the best 
transformation is kept Cham et al, extends this method by, at each step, choosing either a 
Kalman filter, a Bayesian RANSAC and a standard RANSAC method. In Bayesian RANSAC, 
each candidate pair is sampled with a prior probability estimated from the likelihood of 
matching, and the candidate transformation calculated using a prior probability from iterations 
performed at coarser scales. 

Thus Cham et al. does not disclose that the probabilities of all the candidate motions for 
each of a plurality of sampling points in the image are kept, and used in the update of the motion 
field. There is no concept of this at all in Cham et al. Further, in Cham et al. the candidate 
matches between features are randomly selected from all possible pairs of feature points, 
whereas in claims 24, 42 and 43 the candidate motions are a set of neighbouring sampling points, 
e..g., local to the location. 

It should also be noted that in Cham et al. the transformation that is estimated (in order to 
perform the mosaicing) is a global, rigid transformation, rather than a local, non-rigid 
deformation* 

Finally, because Cham et al- relies on the use of a feature detector, it would not work if 
features are not present. It will only work if the feature detector finds a reasonable set of 
corresponding points (say 1,000 points with 30% correct matches), whereas claims 24, 42 and 43 
can work, by way of example, on an arbitrary (regular grid) set of sampling points set across the 
image. This is because the claimed method and apparatus can permit detecting of movement in 
images where features cannot easily be recognized. 

Thus, even disregarding the fact that Cham et al. is concerned with rigid transformations, 
Cham et al. does not disclose the feature found in the amended independent claims that for each 
of the sampling points there are a plurality of stored motion probabilities, and that each of these 
is iteratively updated on the basis of motion probabilities selected from the plurality stored for 
each of the neighboring sampling points. 
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In view of the fact that Martens was cited only to demonstrate extensions of motion field 
estimation to non-rigid images, Applicants respectfully submit that it is clear that Martens does 
not disclose the feature introduced into the amended independent claims. For at least these 
reasons, Applicants submit that the proposed combination of Cham et al. and Martens does not 
render the subject matter of claims 24, 42 and 43, or their rejected dependent claims, obvious. 

Claims 33 and 34 were rejected under 35 U.S.C. Section 103(a) as allegedly being 
"obvious" over the proposed Cham et al.-Martens combination, in further view of Iu (EP 0 652 
536). Iu is cited for its purported disclosure relating to the rejection of outliers. Even assuming 
that Iu is viewed in this manner and that proper motivation could be identified for Iu's 
combination with Cham et al. and Martens, Iu does not remedy the above-noted deficiencies of 
Cham et al. and Martens with respect to claim 24, from which claims 33 and 34 depend. For at 
least these reasons, claims 33 and 34 are believed to be allowable. 

Claims 40 and 41 were rejected under 35 U.S.C. Section 103(a) as allegedly being 
"obvious" over the proposed Cham et al.-Martens combination, in further view of Harms (U.S- 
Patent No. 5,214,382). Harms is cited for its disclosure of obtaining an image of a breast using 
MRI and a contrast agent. Harms does not remedy the above-noted deficiencies of Cham et al- 
and Martens with respect to claim 24 (from which claims 40 and 41 depend) and thus Harms' 
addition to the Cham et aL-Marten$ combination would not result in the subject matter of claims 
40 and 41. 

New claim 44 has been added. This claim depends from claim 24 and is believed to be 
allowable because of this dependency. 
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The pending claims are believed to be allowable and favorable office action is 
respectfully requested. 

Respectfully submitted, 



NIXON & VANDERHYE P.C. 




MJS:dbp 

1100 North Glebe Road, 8th Floor 
Arlington, VA 22201-4714 
Telephone: (703) 816-4000 
Facsimile: (703)816-4100 
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